Video quality testing by encoding aggregated clips

ABSTRACT

A variable-bit-rate (VBR)-encoding of a plurality of clips from a plurality of video content items is performed to produce a VBR-encoded aggregated video content item. The VBR-encoding of each of the clips is based on an encoding complexity of at least one other one of the clips. This can be performed by combining the clips into aggregated video content and two-pass VBR-encoding the aggregated video content. A video quality test may be performed using the VBR-encoded aggregated video content item.

FIELD OF THE DISCLOSURE

The present disclosure relates to video quality testing.

BACKGROUND

Variable-bit-rate (VBR) encoding refers to a video encoder's ability tovary an amount of data used to encode a scene based on an overallcomplexity of information being encoded. A video scene with a largeamount of detail and movement, for example, may be encoded using moredata than a scene that is relatively simple and lacks motion.

Pre-encoding refers to encoding an entire video before the video isdelivered. Pre-encoding may involve an encoder analyzing the entirevideo to be encoded to generate a more sophisticated encoding. Whenencoding a movie, for example, the entire movie can be analyzed todetermine which parts of the movie are relatively complex and whichparts are relatively simple. The encoder can pre-allocate, from anoverall data budget, a specific amount of data for each section of themovie. Such an approach allows the encoder to maintain a limit on theoverall size of an output file while optimizing the allocation of datato specific parts of the movie. Video quality can be optimized withinthe constraint of an overall file size.

The combination of VBR and pre-encoding enables higher-quality video(when compared to other encoding mechanisms) to be delivered using thesame amount of data. Pre-encoded VBR content is a common type of encodedcontent for many download-to-play video services found on the Internet.Encoding schemes and/or video distribution and display systems areevaluated using video quality and acceptability test procedures. Theseprocedures often involve multiple pieces of video content to provide adiverse set over which to test.

BRIEF DESCRIPTION OF THE DRAWINGS

The present invention is pointed out with particularity in the appendedclaims. However, other features are described in the following detaileddescription in conjunction with the accompanying drawings in which:

FIG. 1 is a flow chart of an embodiment of a method of performing avideo quality test; and

FIG. 2 is a block diagram of an embodiment of a system for performingthe video quality test.

DETAILED DESCRIPTION OF THE DRAWINGS

The multiple pieces of video content used in a video quality test maycomprise multiple full-length movies, for example. For purposes ofillustration and example, consider a video quality test that involvesthirty full-length movies having an average length of roughly ninetyminutes per movie. To test multiple different encoding schemes, asignificant amount of time is required to encode the multiplefull-length movies in each of the encoding schemes.

To save money on costs of encoding in a video quality test, a singleclip from each movie may be encoded rather than the entire movie. Forexample, a three-minute clip may be encoded rather than an entireninety-minute movie to reduce an amount of encoding by a factor of about30×. However, for testing pre-encoded VBR schemes, a first amount ofdata from an encoding of the three-minute clip by itself is notnecessarily the same as a second amount of data allocated to thethree-minute clip from an encoding of the entire movie. An analysis oftest data has shown that the first and second amount of data cansubstantially differ, in practice.

Embodiments of the present disclosure address this problem withouthaving to resort to encoding all thirty movies in their entirety. Themultiple clips from the different movies are combined into a single,aggregated piece of video content. VBR pre-encoding, such as two-passVBR encoding, is performed on the aggregated piece of video content toproduce a VBR pre-encoded aggregated video content item. By encoding theaggregated piece of video content in this manner, each clip is encodedwithin a larger context of the other clips. As a result, more complexclips are encoded using more data while simpler clips are encoded usingless data. The total amount of data for encoding the aggregated clipscan be the same as for individually encoding the clips, but thedistribution of data can vary from clip-to-clip similar to if the clipswere taken from fully-encoded movies. Thus, the VBR pre-encodedaggregated video content item has encoded clips that more closely matchclips from full-length encodings without requiring the full-lengthencodings to be generated (e.g. encoding all thirty full-length movies).As a result, a meaningful video quality test can be performed for anencoding of 90 minutes (30 movies×3 minutes/movie) of data rather than30 full-length movies of data that normally would have been required.

Embodiments of the present disclosure are described with reference toFIG. 1, which is a flow chart of an embodiment of a method of performinga video quality test, and FIG. 2, which is a block diagram of anembodiment of a system for performing a video quality test.

As indicated by block 10, the method comprises VBR pre-encoding anaggregation of a plurality of clips from a plurality of video contentitems. This act produces a VBR pre-encoded aggregated video contentitem. Because the aggregation is being VBR pre-encoded, each of theclips is encoded based on an encoding complexity of at least one otherone of the clips. In some embodiments, each of the clips is VBR-encodedbased on an encoding complexity of the clip relative to encodingcomplexities of some or all others of the clips. In one embodiment, theact of VBR pre-encoding comprises performing a two-pass VBR-encoding ofa single, aggregated piece of video content that concatenates orotherwise combines the plurality of clips. The two-pass VBR-encodingincludes a first pass that analyzes the entire single, aggregated pieceof video content, and a second pass that encodes the single, aggregatedpiece of video content based on the analysis.

As described above, the plurality of video content items may comprise aplurality of movies. The movies may comprise cinematic movies,made-for-television movies or other movies produced for a mass audience,for example. The movies may be individually-released and/orindividually-purchasable movies. In these cases, each movie normally canbe purchased, rented, downloaded or viewed independently of the othermovies.

In some embodiments, each clip is of a duration substantially less thana duration of its source video content item. As described above, theduration of a clip may be about three minutes relative to aninety-minute source video content item.

The duration of each clip is selected to be long enough so that humansubjects in a video quality test can become psychologically involved inviewing the clip. However, the duration of each clip is selected to beshort enough to avoid prolonging the amount of time each human subjectis watching clips and the amount of time needed to encode the aggregatedclips. In some embodiments, the duration of each clip is within a rangeof about one minute to about two minutes. In other embodiments, theduration of each clip is within a range of about one minute to aboutthree minutes.

In some embodiments, some, most or all of the clips have about the sameduration. As described above, some, most or all of the clips may have aduration of about three minutes. In alternative embodiments, the clipsmay have different durations.

The number of clips that are concatenated to form the single, aggregatedpiece of video content is selected to provide a sufficiently-largesample size to perform a statistical analysis of the human subjects'evaluations of overall quality. However, the number of clips is selectedto be small enough to avoid prolonging the amount of time each humansubject is watching clips and the amount of time needed to encode theaggregated clips. In some embodiments, the number of clips is within arange of about ten clips to about fifty clips, with thirty clips beingused in one embodiment.

For purposes of illustration and example, consider four clips 12, 14, 16and 18 from four video content items 22, 24, 26 and 28, respectively, asshown in FIG. 2. A combiner 30 concatenates or otherwise combines thefour clips 12, 14, 16 and 18 into a single, aggregated piece of videocontent 32. A VBR encoder 33 VBR-encodes the aggregated piece of videocontent 32 to produce a VBR pre-encoded aggregated video content item34. The VBR encoder 33 may perform a two-pass VBR encoding of theaggregated piece of video content 32 to produce the VBR pre-encodedaggregated video content item 34.

After the encoding has completed, the method comprises performing avideo quality test using the VBR-encoded aggregated video content item34, as indicated by block 36. The video quality test is performed by avideo quality test apparatus 40 that decodes and plays back theVBR-encoded aggregated video content item 34. The video quality test maybe performed using human subjects, who view a display of the decodedcontent and provide one or more subjective ratings of the video quality.Alternatively, the video quality test may be automated using a computerthat rates characteristics of the decoded content.

The acts indicated by blocks 10 and 36 may be repeated for multipledifferent VBR encoding schemes, parameters and modes. The resultingmultiple video quality tests can be analyzed to determine desirable VBRencoding scheme(s), parameter(s) and mode(s) based on one or more videoquality objectives and one or more constraints (e.g. bandwidth or bitrate constraints).

In addition to quality testing, the herein-disclosed encoding method andsystem can be used to enhance the video quality of commercials embeddedin longer content items such as television programs or movies.Separately encoding the longer content and the commercials can result inquality discontinuities from content-to-commercials and/or fromcommercials-to-content. For example, if a relatively complex-to-encodecommercial is embedded in a relatively simple-to-encode program, a totalamount of data available for a relatively short commercial may beinsufficient to encode the commercial with a desirable level of quality.Encoding the aggregated program and commercial would cause the programto be encoded with less data than if encoded individually. The resultingunused data can be applied to the commercial. The resulting quality ofthe commercial is higher with only a small degradation to the rest ofthe program. Further, the viewer experiences a more consistent videoquality.

The herein-described components may be embodied by one or more computerprocessors directed by computer-readable program code stored by acomputer-readable medium.

Referring to FIG. 3, an illustrative embodiment of a general computersystem is shown and is designated 300. The computer system 300 caninclude a set of instructions that can be executed to cause the computersystem 300 to perform any one or more of the methods or computer basedfunctions disclosed herein. The computer system 300 may operate as astandalone device or may be connected, e.g., using a network, to othercomputer systems or peripheral devices.

In a networked deployment, the computer system may operate in thecapacity of a server or as a client user computer in a server-clientuser network environment, or as a peer computer system in a peer-to-peer(or distributed) network environment. The computer system 300 can alsobe implemented as or incorporated into various devices, such as apersonal computer (PC), a tablet PC, a set-top box (STB), a personaldigital assistant (PDA), a mobile device, a palmtop computer, a laptopcomputer, a desktop computer, a communications device, a wirelesstelephone, a land-line telephone, a control system, a camera, a scanner,a facsimile machine, a printer, a pager, a personal trusted device, aweb appliance, a network router, switch or bridge, or any other machinecapable of executing a set of instructions (sequential or otherwise)that specify actions to be taken by that machine. In a particularembodiment, the computer system 300 can be implemented using electronicdevices that provide voice, video or data communication. Further, whilea single computer system 300 is illustrated, the term “system” shallalso be taken to include any collection of systems or sub-systems thatindividually or jointly execute a set, or multiple sets, of instructionsto perform one or more computer functions.

As illustrated in FIG. 3, the computer system 300 may include aprocessor 302, e.g., a central processing unit (CPU), a graphicsprocessing unit (GPU), or both. Moreover, the computer system 300 caninclude a main memory 304 and a static memory 306, that can communicatewith each other via a bus 308. As shown, the computer system 300 mayfurther include a video display unit 310, such as a liquid crystaldisplay (LCD), an organic light emitting diode (OLED), a flat paneldisplay, a solid state display, or a cathode ray tube (CRT).Additionally, the computer system 300 may include an input device 312,such as a keyboard, and a cursor control device 314, such as a mouse.The computer system 300 can also include a disk drive unit 316, a signalgeneration device 318, such as a speaker or remote control, and anetwork interface device 320.

In a particular embodiment, as depicted in FIG. 3, the disk drive unit316 may include a computer-readable medium 322 in which one or more setsof instructions 324, e.g. software, can be embedded. Further, theinstructions 324 may embody one or more of the methods or logic asdescribed herein. In a particular embodiment, the instructions 324 mayreside completely, or at least partially, within the main memory 304,the static memory 306, and/or within the processor 302 during executionby the computer system 300. The main memory 304 and the processor 302also may include computer-readable media.

In an alternative embodiment, dedicated hardware implementations, suchas application specific integrated circuits, programmable logic arraysand other hardware devices, can be constructed to implement one or moreof the methods described herein. Applications that may include theapparatus and systems of various embodiments can broadly include avariety of electronic and computer systems. One or more embodimentsdescribed herein may implement functions using two or more specificinterconnected hardware modules or devices with related control and datasignals that can be communicated between and through the modules, or asportions of an application-specific integrated circuit. Accordingly, thepresent system encompasses software, firmware, and hardwareimplementations.

In accordance with various embodiments of the present disclosure, themethods described herein may be implemented by software programsexecutable by a computer system. Further, in an exemplary, non-limitedembodiment, implementations can include distributed processing,component/object distributed processing, and parallel processing.Alternatively, virtual computer system processing can be constructed toimplement one or more of the methods or functionality as describedherein.

The present disclosure contemplates a computer-readable medium thatincludes instructions 324 or receives and executes instructions 324responsive to a propagated signal, so that a device connected to anetwork 326 can communicate voice, video or data over the network 326.Further, the instructions 324 may be transmitted or received over thenetwork 326 via the network interface device 320.

While the computer-readable medium is shown to be a single medium, theterm “computer-readable medium” includes a single medium or multiplemedia, such as a centralized or distributed database, and/or associatedcaches and servers that store one or more sets of instructions. The term“computer-readable medium” shall also include any medium that is capableof storing, encoding or carrying a set of instructions for execution bya processor or that cause a computer system to perform any one or moreof the methods or operations disclosed herein.

In a particular non-limiting, exemplary embodiment, thecomputer-readable medium can include a solid-state memory such as amemory card or other package that houses one or more non-volatileread-only memories. Further, the computer-readable medium can be arandom access memory or other volatile re-writable memory. Additionally,the computer-readable medium can include a magneto-optical or opticalmedium, such as a disk or tapes or other storage device to capturecarrier wave signals such as a signal communicated over a transmissionmedium. A digital file attachment to an e-mail or other self-containedinformation archive or set of archives may be considered a distributionmedium that is equivalent to a tangible storage medium. Accordingly, thedisclosure is considered to include any one or more of acomputer-readable medium or a distribution medium and other equivalentsand successor media, in which data or instructions may be stored.

Although the present specification describes components and functionsthat may be implemented in particular embodiments with reference toparticular standards and protocols, the invention is not limited to suchstandards and protocols. For example, standards for Internet and otherpacket switched network transmission (e.g., TCP/IP, UDP/IP, HTML, HTTP)represent examples of the state of the art. Such standards areperiodically superseded by faster or more efficient equivalents havingessentially the same functions. Accordingly, replacement standards andprotocols having the same or similar functions as those disclosed hereinare considered equivalents thereof.

The illustrations of the embodiments described herein are intended toprovide a general understanding of the structure of the variousembodiments. The illustrations are not intended to serve as a completedescription of all of the elements and features of apparatus and systemsthat utilize the structures or methods described herein. Many otherembodiments may be apparent to those of skill in the art upon reviewingthe disclosure. Other embodiments may be utilized and derived from thedisclosure, such that structural and logical substitutions and changesmay be made without departing from the scope of the disclosure.Additionally, the illustrations are merely representational and may notbe drawn to scale. Certain proportions within the illustrations may beexaggerated, while other proportions may be minimized. Accordingly, thedisclosure and the figures are to be regarded as illustrative ratherthan restrictive.

One or more embodiments of the disclosure may be referred to herein,individually and/or collectively, by the term “invention” merely forconvenience and without intending to voluntarily limit the scope of thisapplication to any particular invention or inventive concept. Moreover,although specific embodiments have been illustrated and describedherein, it should be appreciated that any subsequent arrangementdesigned to achieve the same or similar purpose may be substituted forthe specific embodiments shown. This disclosure is intended to cover anyand all subsequent adaptations or variations of various embodiments.Combinations of the above embodiments, and other embodiments notspecifically described herein, will be apparent to those of skill in theart upon reviewing the description.

The Abstract of the Disclosure is provided to comply with 37 C.F.R.§1.72(b) and is submitted with the understanding that it will not beused to interpret or limit the scope or meaning of the claims. Inaddition, in the foregoing Detailed Description, various features may begrouped together or described in a single embodiment for the purpose ofstreamlining the disclosure. This disclosure is not to be interpreted asreflecting an intention that the claimed embodiments require morefeatures than are expressly recited in each claim. Rather, as thefollowing claims reflect, inventive subject matter may be directed toless than all of the features of any of the disclosed embodiments. Thus,the following claims are incorporated into the Detailed Description,with each claim standing on its own as defining separately claimedsubject matter.

The above disclosed subject matter is to be considered illustrative, andnot restrictive, and the appended claims are intended to cover all suchmodifications, enhancements, and other embodiments which fall within thetrue spirit and scope of the present invention. Thus, to the maximumextent allowed by law, the scope of the present invention is to bedetermined by the broadest permissible interpretation of the followingclaims and their equivalents, and shall not be restricted or limited bythe foregoing detailed description.

1. A method comprising: variable-bit-rate (VBR)-encoding a plurality ofclips from a plurality of video content items to produce a VBR-encodedaggregated video content item, wherein the VBR-encoding of each of theclips is based on an encoding complexity of at least one other one ofthe clips; and performing a video quality test using the VBR-encodedaggregated video content item.
 2. The method of claim 1 wherein theVBR-encoding of each of the clips is based on an encoding complexity ofthe clip relative to encoding complexities of others of the clips. 3.The method of claim 1 wherein said VBR-encoding comprises performing atwo-pass VBR-encoding of a single, aggregated piece of video contentthat combines the plurality of clips.
 4. The method of claim 1 whereinthe plurality of video content items comprise a plurality ofindividually-purchasable movies.
 5. The method of claim 1 wherein mostof the plurality of clips have about the same duration.
 6. The method ofclaim 5 wherein the same duration is about three minutes.
 7. A method ofencoding a video program having a plurality of program segments and atleast one embedded commercial segment, the method comprising:variable-bit-rate (VBR)-encoding the video program with the plurality ofprogram segments and the at least one embedded commercial segment toproduce a VBR-encoded video program item, wherein the VBR-encoding ofthe at least one commercial segment is based on an encoding complexityof at least one of the plurality of program segments.
 8. The method ofclaim 7 further comprising: broadcasting the VBR-encoded video programitem.
 9. A system comprising: a variable-bit-rate (VBR)-encoder toVBR-encode a plurality of clips from a plurality of video content itemsto produce a VBR-encoded aggregated video content item, wherein theVBR-encoding of each of the plurality of clips is based on an encodingcomplexity of another one of the plurality of clips; and a video qualitytest apparatus to perform a video quality test using the VBR-encodedaggregated video content item.
 10. The system of claim 9 wherein theVBR-encoding of each of the plurality of clips is based on an encodingcomplexity of the clip relative to encoding complexities of others ofthe clips.
 11. The system of claim 9 wherein the VBR-encoder performs atwo-pass VBR-encoding of a single, aggregated piece of video contentthat combines the plurality of clips.
 12. The system of claim 9 whereinthe plurality of video content items comprise a plurality ofindividually-purchasable and downloadable content.
 13. The system ofclaim 9 wherein most of the clips have about the same duration.
 14. Thesystem of claim 13 wherein the same duration is less than three minutes.